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Abstract: 

Higher education's ability to retain students through to graduation appears to be an international 
challenge. This is also the case in South Africa where only 27% of student complete their studies in 
minimum time and where 55% are unlikely to ever graduate. These challenges have meant that 
extended degree programmes, where degrees are formally done over a longer period of time, have 
become a feature of South African education. One challenge is determining which students will 
benefit from an extended programme. In South Africa there are two sets of assessments that are 
pertinent to this debate: the national school leaving examinations (a statutory requirement for entry 
into higher education) and the National Benchmark Tests. The national school leaving assessments 
are norm-referenced, making itoften difficult to interpret the results for placement purposes. The 
National Benchmark Tests are criterion-referenced, and are thus better suited for this placement. 
This paper describes the two assessments, and tracks the academic standing of a cohort of students 
over six years at one higher education institution. It argues that using the results of the two 
assessments in complementary ways is the most productive approach for the purpose of placement 
at this institution and others in South Africa. The implications of considering these assessments is 
briefly explored in broader higher education contexts. 
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Introduction 

Most higher education systems internationally struggle with the challenges of low throughput rates 
and high dropout rates. Among the 18 Organisation for Economic Co-operation and Development 
(OECD) countries for which data were available in 2010, some 31% of students who enter higher 
education fail to graduate. Graduation rates differ widely by country with some (Mexico, New 
Zealand, Sweden and the United States) having more than 40% of their higher education students 
not graduating, whereas for others (Belgium (FI), Denmark, France, Japan, Korea, Spain and the 
Russian Federation) this proportion is less than 25% (OECD 2010). 

For South African higher education only 27% graduate in minimum time, the majority of students 
take up to two years more than the minimum time for their degree studies and 55% of them never 
graduate (Scott et. al. 2007, Council on Higher Education 2013). These low throughput rates and high 
dropout rates exacerbate the problem of students taking longer than the minimum time. Extended 
degree programmes structure the curricula in such a manner that students can typically undertake 
their degree studies over the minimum time plus one year. One difficulty is how best to determine 
which students would benefit most from an extended degree programme and which students would 
most likely cope with doing the degree in minimum time. In South Africa there are two assessments 
of school-leavers that are pertinent to this debate. The first is the national school leaving 
examination, the National Senior Certificate (NSC), which is a statutory requirement for entry into 
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higher education. The results of the NSC are often difficult to interpret for the purposes of admission 
and placement. The second assessment is the National Benchmark Tests (NBT) (Griesel 2006). Two 
of the NBT Project objectives are a) to provide a service to higher education institutions requiring 
additional information to assist in admission 1 (selection and placement) of students in appropriate 
curricular routes and b) to assist with curriculum development, particularly in relation to foundation 
and augmented courses (Griesel 2006, 4). The NBT through its conceptualisation, design and 
implementation is particularly well placed to provide information for the purpose of placement. 

It is not only the content that is taught in the National Curriculum Statement (the school curriculum) 
that is essential for performing well in South African higher education. More important are the 
cognitive abilities and skills underlying the school curriculum content domains that are learned in 
one context and are believed to be transferable to another. The NBT therefore focus on the 
application of knowledge acquired in the school context, in the higher education context. The NBTP 
is thus designed to provide the kind of information which the South African higher education sector 
requires in order to make decisions about the most appropriate curriculum structures for students. 

This paper argues that ranking applicants for selection is important, but there are two even more 
important considerations when making selection decisions. The one is placing students in 
appropriate curriculum structures and the second is that the curriculum structures themselves 
should take into account what students can and cannot do. Without mechanisms for appropriately 
placing students and identifying how the curricula should respond to the students who are admitted, 
the South African higher education sector is likely to continue to fail students. I argue that the 
extended degree programmes, which serve approximately fifteen percent of the first year higher 
education population, should become the norm, and what are often referred to as regular or 
mainstream programmes should progressively become the exception. I use tracking information of 
students who entered three faculties (Commerce, Engineering and the Built Environment and 
Science) at a South African higher education institution for the first time in 2009 and their 
subsequent academic standing information, in terms of whether they have graduated, dropped out 
or are still studying, after six years. I argue that the criterion-referenced information gained through 
the NBTP could fruitfully complement the NSC norm-referenced assessment to address the main 
challenge for South African higher education, which is to find a mechanism to place students in the 
two curricula routes that is both credible and valid. 

The South African higher education context 

South African higher education is marked by low graduation rates and high levels of failure and 
dropout. These continue to affect the historical apartheid 'African', 'Coloured' and 'Indian' racial 
groupings the most. Thus far the basis on which South African higher education admissions decisions 
have been made has largely been performance on the NSC assessments. The consequences of this 
have been in part racialized patterns of participation, where 60% of the 20-24 White age group 
participate while only 12% of both Coloured and African in the 20-24 age groups participate (Scott 
et. al. 2007). The graduation and dropout rates equally reflect the racialized patterns with White 
students being 1.8 times more likely to complete a Mathematical Sciences degree within five years 
than their African counterparts, and White students being 2.5 times more likely to graduate with a 
Business or Management degree within five years than their African counterparts (Scott et. al. 2007, 
16). The current admission requirements and curricula appear to reproduce the historical apartheid 
patterns. 

If South African higher education is to seriously attempt to change these patterns it will have to 
ensure that curricula are better aligned to student needs. This is not to say that standards should be 
reduced, but rather that standards should be maintained so that students are able to navigate their 


1 Flere selection has the usual meaning and placement means being placed on extended programmes rather 
than being placed on completely different programmes as it might be understood. 
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studies in a manner that best suits their levels of preparedness. Higher education institutions have 
autonomy over their admission criteria and admissions decisions are often sites of contestation. 
Admission criteria as well as the assumptions underlying these criteria and its consequences for both 
admission and teaching and learning need to be questioned, but it is not the purpose of the current 
paper to do so. The manner in which the South African education sector reproduces the historical 
inequalities and how the higher education admission criteria feeds into this should also be 
questioned. Admissions decisions must take into account the manner in which schooling is racialized 
and that racial classifications continue to play out in terms of higher education participation, 
graduation and drop-out rates. 

A number of mistaken views still prevail. Some myths that need to be dispelled are that students 
take three years for three year degree programmes and four years for four year degree programmes 
(only 27% of first time entrants do so); that extended degree programmes are for the minority and 
are inferior to the regular programmes. The higher education sector has much to do to address the 
stigma associated with extended degree programmes, since there are often notions of deficit 
attached to those students who could benefit from these programmes. There is a need for both 
students and institutions who are reluctant to embrace extended degree programmes to have a 
sober look at the realities and to consider the admission and curricular options that would best serve 
the students, individual higher education institutions, and the sector as a whole. 

The criterion-referenced and the norm-referenced assessments 

The two sets of assessments in the South African national landscape are fundamentally different in 
terms of purpose, intentions, design and manner of delivery. The National Benchmark Tests are 
criterion-referenced, which means that they are "constructed to provide information about the level 
of a test-taker's performance in relation to clearly defined domains of content and/or behaviours 
(e.g. reading, writing, mathematics) that require mastery" (Foxcroft 2006, 9) while the National 
Senior Certificate examinations are norm-referenced, which means that performance in a particular 
year is normed to a "norm group". Norm-referenced tests are designed to sort and rank students, 
usually 'on a curve', not to see if they meet a standard or criterion. The National Senior Certificate 
attempts to answer the question whether scholars are ready to exit the school system while the 
National Benchmark Tests attempt to answer the question how ready are prospective higher 
education students for the demands of higher education. 

The criterion-referenced National Benchmark Tests 

The NBTs are based on academically researched test specifications and use modern test theories 
(Yen and Fitzpatrick 2006) to determine test scores. They use criterion-referenced benchmarks set 
through standard setting methods to place candidates in proficiency bands which describe both their 
preparedness for the demands of higher education and the extent to which the curricula should be 
responsive to the preparedness of the candidates that are admitted. The NBTs assess students and 
prospective students in the three domains of Academic Literacy; Quantitative Literacy and 
Mathematics in the languages of instruction, namely English and Afrikaans. 

The Academic Literacy test assesses a students' capacity to engage successfully with the language 
demands of academic study. The test is an assessment of the generic academic reading and 
reasoning ability of prospective applicants. The construct of academic literacy (Cliff and Yeld 2006) 
on which the test is based has a well-theorised history (Bachman and Palmer 1996; Cummins 2000; 
Yeld 2001; Cliff, Yeld and Hanslo 2003) and empirical studies have been reported exploring 
associations between performance on this construct and academic performance in a wide range of 
South African higher education contexts (Cliff, Ramaboa and Pearce 2007; Cliff and Hanslo 2009). 

The Quantitative Literacy test assesses a students' ability to manage situations or solve problems of 
a quantitative (mathematical and statistical) nature in real contexts relevant to quantitative 
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disciplines in higher education (Prince and Archer 2006; Prince and Archer 2008; Prince and Simpson 
2016). The definition of quantitative literacy and the construct which underpins the NBT 
Quantitative Literacy test (Frith and Prince 2006; Frith and Prince 2009) is strongly influenced by the 
definition of numerate behaviour underlying the assessment of numeracy in the Adult Literacy and 
Lifeskills (ALL) Survey (Gal, van Groenestijn, Manly, Schmitt & Tout 2005, 152) and the New Literacies 
Studies' view of literacy as social practice (Street 2005; Street & Baker 2006; Kelly, Johnston & 
Baynham 2007). 

While the Academic Literacy and Quantitative Literacy assessments are generic in nature the 
Mathematics test assesses a students' manifest ability related to mathematical concepts formally 
part of the school Mathematics curriculum relevant to the subject Mathematics itself and 
mathematical disciplines such as Physics and Chemistry. The Mathematics assessment therefore 
assesses the degree to which learners have achieved the ability to manipulate, synthesise a number 
of different mathematical concepts, and draw strictly logical conclusions in abstract symbolic 
contexts (Bohlmann and Braun 2006). These higher-order skills underlie success in higher education 
mathematics. 

The Academic Literacy, Quantitative Literacy and Mathematics assessments make use of multiple 
choice items that are mapped onto the respective test specification tables. Responses are scored 
using the unidimensional three-parameter (a, b, c) Item Response Theory (IRT) model, where a = 
discrimination, b = difficulty, and c = guessing/pseudo-chance (Yen and Fitzpatrick 2006,114). 
Academic Literacy, Quantitative Literacy and Mathematics items are scored dichotomously, that is 
either as right or wrong. All three assessments contain common items and modern test theories are 
used to ensure that the scores on different versions of the assessments are linked and equated 
(Holland and Dorans 2006) to ensure that performance on different versions of the test is 
comparable and is not a function of the version of the test that the candidate has written. The 
standard-setting method that is employed to determine the benchmarks is the modified Angoff 
method (Hambleton and Pitoniak 2006, 435). Table 1 provides a description of the Academic Literacy 
(AL), Quantitative Literacy (QL) and Mathematics (MAT) performance standards defining proficiency 
levels for degree study, the score ranges and suggested institutional responses to candidates 
performing at these levels. Performance standards have also been set for Diploma and Higher 
Certificate study and these are different to the ones for Degree study. 
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Table 1. Nationa 

1 Benchmark Test degree performance standards and their interpretations. 

Performance 

band 

Score Range 

Description 

Proficient 

AL: 64%-100% 

QL: 70%-100% 

MAT: 68%-100% 

Test performance suggests that future academic 
performance will not be adversely affected (students 
may pass or fail at university, but this is highly unlikely to 
be attributable to strengths or weaknesses in the 
domains tested). If admitted, students may be placed 
into regular programmes of study. 

Intermediate 

AL: 38%-63% 

QL: 38%-69% 

MAT: 35%-67% 

The challenges identified are such that it is predicted 
that academic progress will be adversely affected. If 
admitted, students' educational needs should be met as 
deemed appropriate by the institution (e.g. extended or 
augmented programmes, special skills provision). 

Basic 

AL: 0%-37% 

QL: 0%-37% 

MAT: 0%-34% 

Test performance reveals serious learning challenges: it 
is predicted that students will not cope with degree-level 
study without extensive and long-term support, perhaps 
best provided through bridging programmes (i.e. non¬ 
credit preparatory courses, special skills provision) or 
further education and training provision. Institutions 
admitting students performing at this level would need 
to provide such support. 


Since the majority of the South African higher education cohort scores have placed them in the 
Intermediate band, it has been found productive to divide the Academic Literacy (AL), Quantitative 
Literacy (QL) and Mathematics (MAT) intermediate performance band into two bands, the 
Intermediate Upper and Lower bands as shown in Table 2. It is important to note that this division 
was not done through the standard-setting exercise but rather through taking the interval mean 
values. 


Table 2. National Benchmark Test degree Intermediate performance standards and their 
interpretations. 

Intermediate 
performance band 

Score Range 

Description 

Intermediate Upper 

AL:51%-63% 

QL:54%-69% 

MAT:52%-67% 

Students are likely to need complementary support 
(additional tutorials, workshops, augmented courses, 
language intensive work) 

Intermediate Lower 

AL:38%-50% 

QL:38%-53% 

MAT:35%-51% 

Students need to be placed in an extended degree 
programme 


I have described the NBTs as criterion-referenced assessments which aim to deliver information 
against benchmarked categories of performance for formal study at institutions of higher learning. I 
will now describe the norm-referenced school-leavers assessment, the National Senior Certificate. 

The norm-referenced National Senior Certificate 
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The NSC assessments are norm referenced and therefore cannot easily be used to assess whether 
candidates meet a certain standard in a subject or domain. For the NSC, part of the final score is 
made up of the course mark and then the scores are 'standardised' or 'normed' to the 5-year rolling 
average score for each subject. So while a candidate may perform well compared to the norm, they 
may still fail to meet a certain standard in the domain being tested. 

The Department of Basic Education approved the achievement scale for NSC subjects (DBE 2011, 9) 
given in Table 3 below. The descriptions against the rating codes are not benchmarks or standards, 
but rather descriptive labels of percentage score ranges. 


Table 3. National Senior Certificate scale of achievement Grades 10-12. 

Rating Code 

Description 

Score 

7 

Outstanding achievement 

80-100 

6 

Meritorious achievement 

70-79 

5 

Substantial achievement 

60-69 

4 

Adequate achievement 

50-59 

3 

Moderate achievement 

40-49 

2 

Elementary achievement 

30-39 

1 

Not achieved 

0-29 


On completing the NSC, a candidate can qualify for higher certificate, diploma or degree study. Table 
4 describes the criteria, from the Department of Basic Education (DBE) (2008, 8), for entry into 
Higher Certificate, Diploma or Degree Study that are used in South Africa. 


Table 4. National Senior Certificate Criteria for Higher certificate, Diploma and Degree study. 

Qualification 

Minimum Entry requirement 

Higher Certificate 

Pass NSC with at least rating of 2 (30-39%) for the Language of Learning 
and Teaching of higher education institution 

Diploma 

Pass NSC with an achievement rating of 3 (40-49%) or better in four 
subjects. At least rating of 2 (30-39%) for the Language of Learning and 
Teaching of the higher education institution. 

Bachelor Degree 

Pass NSC with an achievement rating of 4 (50-59%) or better in four 
subjects from the designated list. At least rating of 2 (30-39%) for the 
Language of Learning and teaching of the higher education institution. 


The Department of Basic Education employs some form of standard setting (DBE op.cit., 19), with 
the question papers set by a panel of 3-5 members representative of as many provinces as possible 
and with the right subject expertise. The department also engages in language simplification by 
checking correlations between Afrikaans and English versions of the question papers. The 
comparisons of the two assessments are summarised in Table 5. 
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Table 5. Comparison of the National Senior Certificate (NSC) and National Benchmark Tests 
(NBT) Assessments. 


National Senior Certificate 
(NSC) Assessments 

National Benchmark Tests 
(NBT) Assessments 

The purpose of the 
assessments 

To what extent do NSC 

candidates meet the 

curriculum statement 
expectations as expressed in 
the Subject Assessment 
Guidelines? 

To what extent do students 
aiming to enter higher 
education meet the core 
academic literacy, quantitative 
literacy and mathematics 
competencies required by 
schoolleavers on entry to 
higher education study? 

The assessment type 

Norm-referenced achievement 

assessments 

Criterion-referenced 
proficiency assessments 

The measurement theory 
employed by the 
assessments 

Classical Test Theory: 

Modern Test Theory: 

Unidimensional three- 
parameter (a, b, c) Item 
Response Theory (IRT) model, 
where a is discrimination, b is 
difficulty, and c is 
guessing/pseudo-chance 

Score ranges: 

0% -100% 

0% -100% 

The Rating/Proficiency 

Levels employed and how 
these are determined 

Rating codes of 1 - 7 
determined through dividing 
performance into 0-29 % for 
band 1, 30-39 %, 40-49 %, ... 
70-79 % for bands 2 to 6 and 
80-100% for band 7 

Proficiency levels of 
'Proficient', 'Intermediate' and 
'Basic' determined through the 
modified Angoff standard 
setting method and the 
'Intermediate' proficiency level 
split into 'Upper' and 'Lower' 
using the midpoint between 
the Basic/Intermediate score 
and the 

Intermediate/Proficient score. 

The number of times the 

assessment can be taken in 
one assessment cycle 

Candidates can only write the 
NSC assessments once at the 
end of the school year 

Candidates can write the NBTs 
twice during any one higher 
education intake cycle 

How the assessment scores 

are determined and 
whether they are 
comparable from one 
assessment instance to the 

next 

In-class assessment scores are 

combined with examination 

scores to arrive at the final 

assessment score 

Assessments have common 

items embedded into them. 
Responses to items on the 
assessments are used to 

determine raw scores and the 

common items are used to link 
and equate the assessments 


This brief overview of differences between the two assessment systems explicate the concept of the 
complementary nature of a norm-referenced assessment such as the NSC and criterion-referenced 
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tests such as the NBTs. Their combined value in informing admission decisions and particularly 
decisions about placement into extended and regular degree curriculum structures will now be 
explored using an institutional case study. 


The case of three faculties at a South African higher education institution 

The NBTP is by design an attempt to address the throughput challenges faced by higher education in 
South Africa. When institutions admit students with the full knowledge of their levels of academic 
preparedness it is imperative that appropriate teaching and learning environments be created. 
However, too often higher education institutions are not sufficiently responsive to the needs of the 
students and in the process students have a particularly bad teaching and learning experience and 
may even drop out. To investigate how the two tests together could predict first time entrants' 
performance at the end of six years, I track the 2 566 first time entering students who entered three 
faculties (Commerce, Engineering and the Built Environment, and Science) at a South African 
University in 2009 over six years, and classify their last academic standing code into one of three 
categories, 'Fail', 'Continuing' and 'Graduate'. The academic standing code is a code of achievement 
(or performance) at the end of each year at this particular higher education institution. Of the 2 566 
first time entrants 2 375 wrote the NBT Academic Literacy and Quantitative Literacy assessment and 
2 253 wrote the NBT Mathematics assessment at the beginning of their studies in 2009. Of the 2 566 
first time entering students 2 035 wrote the NSC English assessment (designated by EN, either Home 
Language or First Additional Language, and for this study performance on these two assessments 
were treated as equivalent) and 2 027 wrote the NSC Mathematics assessment (designated by 
MTHN). 

After providing the background characteristics of the entire sample, I present the overall distribution 
of scores for those who wrote the two NSC and three NBT assessments. In addition, I report on the 
academic standing of the sample of students after six years and the associations between 
performance on the two NSC and three NBT assessments and academic standing for the cohort of 
students who wrote these assessments. 

Some self-reported demographic characteristics of this sample are shown in Table 6. The majority of 
candidates were African (39.95%) and the majority speak English (62.04%) as their home language. 
There were more male students (56.27%) than females in this sample. 
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Table 6. Demographic characteristics of the 2 566 First Time Entrants. 


number 

% 

Gender 



Male 

1444 

56.27% 

Female 

1 119 

43.61% 

Not specified 

3 

0.12% 

Population group 



African 

1 025 

39.95% 

Coloured 

305 

11.89% 

Indian 

322 

12.55% 

White 

890 

34.68% 

Not specified 

24 

0.94% 

Home language 



Afrikaans 

60 

2.34% 

English 

1 592 

62.04% 

isiNdebele 

9 

0.35% 

isiXhosa 

250 

9.74% 

isiZulu 

218 

8.50% 

Sesotho 

73 

2.84% 

Sesotho sa Leboa 

464 

7.3% 

Setswana 

79 

3.08% 

siSwati 

32 

1.25% 

Tshivenda 

55 

2.14% 

Xitsonga 

27 

1.05% 

Other 

97 

3.78% 


Table 7 shows how the students performed on the two NSC assessments and Table 8 shows how the 
scores of the sample of students were distributed among the NSC rating codes. 


Table 7. Descriptive statistics for the First Time Entrants on the NSC assessments. 

Assessment 

n 

Mean 

SD 

Minimum 

25 th 

percentile 

Median 

75 th 

percentile 

Maximum 

EN 

2034 

73.07 

9.20 

43 

67 

73 

80 

96 

MTHN 

2027 

83.94 

9.59 

45 

78 

85 

91 

100 


30 





PRINCE: PREDICTING SUCCESS IN HIGHER EDUCATION: THE VALUE OF CRITERION AND NORM- 

REFERENCED ASSESSMENTS 


Table 8. Frequencies of performance for the First Time Entrants on the NSC assessments. 

Rating code 


NSC EN 

NSC MTHN 


n 

% 

n 

% 

3: Moderate achievement 

11 

0.54 



4: Adequate achievement 

148 

7.28 

27 

1.33 

5: Substantial achievement 

545 

26.79 

120 

5.92 

6: Meritorious achievement 

795 

39.09 

442 

21.82 

7: Outstanding achievement 

535 

26.30 

1437 

70.93 

Total 

2034 

100.00 

2253 

100.00 


It is clear that the majority (65.39%) were classified as having achieved the NSC at either Meritorious 
or Outstanding levels in English, while nearly all (92.74%) were classified as having achieved the NSC 
at either Meritorious or Outstanding levels in Mathematics. These results would suggest that the 
majority of these students would have been able to cope with the Language, Quantitative Literacy 
and Mathematics demands of higher education. 

Table 9 shows how the students performed on the NBT assessments of Academic Literacy (AL), 
Quantitative Literacy (QL) and Mathematics (MAT). 


Table 9. Descriptive statistics for the First Time Entrants on the NBT assessments. 

Domain 

n 

Mean 

SD 

Minimum 

25 th percentile 

Median 

75 th percentile 

Maximum 

AL 

2375 

68.23 

11.80 

26 

61 

70 

78 

92 

QL 

2375 

62.31 

17.05 

14 

49 

64 

76 

96 

MAT 

2253 

48.78 

11.85 

0 

40 

48 

55 

80 


Table 10 shows how the scores of the sample of students were distributed among the Academic 
Literacy (AL), Quantitative Literacy (QL) and Mathematics (MAT) proficiency bands. It is clear that the 
majority (67.28%) were classified as proficient in Academic Literacy, while less than half (46.15%) 
were classified as proficient in Quantitative Literacy and only 14.91% were considered proficient in 
Mathematics. Most of the students (76.25%) had scores in the Intermediate band. These results 
suggests that the majority of these students would have needed some kind of extended support in 
Mathematics, more than half would have needed supplementary support in Quantitative Literacy 
and about a third would have required supplementary support in Academic Literacy. 


Table 10. Frequencies of performance for the First Time Entrants on the NBT assessments. 

Benchmark band 

AL 

n % 

SP 

o\ 

_l 

a 

MAT 

n % 

Basic 

60 2.52 

211 8.88 

199 8.83 


31 





PRINCE: PREDICTING SUCCESS IN HIGHER EDUCATION: THE VALUE OF CRITERION AND NORM- 

REFERENCED ASSESSMENTS 


Intermediate Lower 

241 

10.15 

462 19.45 

847 

37.59 

Intermediate Upper 

476 

20.04 

606 25.52 

871 

38.66 

Proficient 

1598 

67.28 

1096 46.15 

336 

14.91 

Total 

2375 

100.00 

2375 100.00 

2253 

100.00 


The academic standing results are presented for the entire sample (n = 2 566) of first time entrants 
into the Commerce, Engineering and the Built Environment, and Science faculties at a South Africa 
university. Table 11 indicate that two thirds of the students graduated and just under a quarter 
failed while 9% are still studying. 


Table 11. Frequencies of Academic standing at the end of six years for the First Time 

Entrants. 

Academic standing 

Frequency 

Percentage 

Graduation (GRAD) 

1719 

66.99% 

Continue (CONT) 

231 

9.00% 

Dropped out (FAIL) 

616 

24.01% 

Grand Total 

2566 

100.00% 


When we look at the association between student performances on the two NSC and three NBT 
assessments at the beginning of their studies with their subsequent academic standing after six 
years we find that it is high for all five assessments. 

From the Pearson Chi-square test statistics in Table 12, there is a statistically significant (simulated 
P<0.001) association between performance in each of the three NBTs, Academic Literacy (AL), 
Quantitative Literacy (QL) and Mathematics (MAT), and the two NSC assessments, English (EN) and 
Mathematics (MTHN), and Academic Standing at the end of six years. The large Chi-square values 
and narrow confidence intervals for the simulated p-values suggest that these associations are 
strong. 


Table 12. Pearson's Chi-squared test for measuring the association between the NSC and NBT 
assessments and Academic Standing categories at the end of six years. 

Pearson's Chi-squared 
test 

NSC 

EN 

NSC MTHN 

NBT 

AL 

NBT 

QL 

NBT MAT 

Chi-square 

125.0658 

127.6819 

150.3343 

174.0292 

156.7539 

two-sided P-value 

2.98E-23 

8.55E-24 

6.58E-30 

6.28E-35 

2.88E-31 

simulated P-value 

0 

0 

0 

0 

0 

99%CI of simulated P- 

value 

0 to 

0.0004604 

11 

0 to 

0.0004604 

11 

0 to 

0.000460411 

0 to 

0.0004604 

11 

0 to 

0.0004604 

11 

Number of simulations 

10000 

10000 

10000 

10000 

10000 


The following five charts in Figures 1 to 5 clearly show these associations, and the statistics 
generated with the Microsoft Excel add-in for the statistical analysis of contingency tables (Slezak et 
al. 2014) in Table 12 support the conclusion that the associations are significant. 
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Academic Standing of the 2009 First Time Entrants 
after 6 years by NSC English rating code. 



■ GRAD 

■ CONT 

■ FAIL 


Figure 1. Percentage of 2009 First Time Entrants in the three academic standing categories by NSC 
English rating codes. 


In Figure 1, except for the level 3 students, graduation (GRAD) patterns at the end of six years 
increased from low to high NSC English results, and failure (FAIL) on the other hand decreased from 
low to high NSC English results at the end of this period. From Table 12, the probability associated 
with the Chi-square statistic of 125.0658 is less than 0.001 and the confidence intervals for the 
simulated p-value are very narrow, suggesting that there is a significant relationship between the 
NSC English assessment levels and Academic Standing variables. 



Figure 2. Percentage of 2009 First Time Entrants in the three Academic Standing categories by NSC 
Mathematics rating codes. 

In Figure 2, graduation (GRAD) and failure (FAIL) patterns at the end of six years were similar for the 
students with NSC Mathematics levels rating codes 4 to 6, and those with NSC Mathematics level 7 
results performed substantially better both in terms of graduation and failure. From Table 12, the 
probability associated with the Chi-square statistic of 127.6818 is less than 0.001 and the confidence 
intervals for the simulated p-value are very narrow, suggesting that there is a significant relationship 
between the NSC Mathematics assessment levels and Academic Standing variables. 
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Academic Standing of the 2009 First Time Entrants 
after 6 years by NBT AL proficiency level. 
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Figure 3. Percentage of 2009 First Time Entrants in the three outcome categories by Academic 
Literacy proficiency bands. 


In Figure 3, graduation (GRAD) patterns at the end of six years increased from low NBT Academic 
Literacy performance levels to high levels and failure (FAIL) on the other hand decreased from low to 
high performance levels in the Academic Literacy test. From Table 12, the probability associated with 
the Chi-square statistic of 150.3343 is less than 0.001 and the confidence intervals for the simulated 
p-value are very narrow, suggesting a significant relationship between the NBT Academic Literacy 
assessment levels and Academic Standing variables. 


Academic Standing of the 2009 First Time Entrants 
after 6 years by NBT QL proficiency level. 
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Figure 4. Percentage of 2009 First Time Entrants in the three outcome categories by Quantitative 
Literacy proficiency bands. 


In Figure 4, graduation (GRAD) patterns at the end of six years increased from low to high NBT 
Quantitative Literacy performance levels. Failure (FAIL), on the other hand, decreased from low to 
high performance levels. From Table 11, the probability associated with the Chi-square statistic of 
174.0292 is less than 0.001 and the confidence intervals for the simulated p-value are very narrow, 
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suggesting that there is a significant relationship between the NBT Quantitative Literacy assessment 
scores and Academic Standing variables. 


Academic Standing of the 2009 First Time Entrants 
after 6 years by NBT MAT proficiency level. 
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Figure 5: Percentage of 2009 First Time Entrants in the three outcome categories by Mathematics 
proficiency bands. 

In Figure 5, graduation (GRAD) patterns at the end of six years increased from low to high NBT 
Mathematics performance levels and failure (FAIL) on the other hand decreased from low to high 
performance levels. From Table 11, the probability associated with the Chi-square statistic of 
156.7539 is less than 0.001 and the confidence intervals for the simulated p-value are very narrow, 
suggesting that there is a significant relationship between the NBT Mathematics assessment scores 
and Academic Standing variables. 

This institutional case study clearly indicates that for this institution performance on the two NSC 
and three NBT assessments are associated with subsequent academic standing and that it would 
have been appropriate for this institution to consider the use of the NBT scores to place students in 
extended degree programmes and to ensure that these students were provided with appropriate 
teaching and learning environments. 

Implications for Higher Education 

The notion of under-preparedness can be applied to both students and higher education institutions. 
Boughey (2009, 4) stresses that the notion of under-preparedness implies 'deficiency' in the 
students only and does not recognise that higher education institutions themselves are 
underprepared to meet the needs of the students that they admit. University admissions and 
teaching need to take into account the capabilities of the students that they accept, place them 
appropriately and make changes to the curriculum to address the 'articulation gap' (Scott, Yeld and 
Hendry 2007, 42) between the demands of curricula and the level of many students' academic 
literacies. In order to design a more responsive curriculum, lecturers and curriculum developers in 
higher education need the kind of information provided by the two different forms of assessment 
about the capabilities of students. 

In order to transform the South African higher education sector it is necessary to admit students 
from educationally disadvantaged backgrounds. Some of the criteria associated with educational 
disadvantage are home and school backgrounds, whether these students are first generation higher 
education students and whether they have gone to private schools or the equivalent. If students are 
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to be admitted into higher education they need to be placed in curricula that are most appropriate 
for their needs. It is for this purpose that e extended curricula programmes exist; the NBTs are well 
suited to identify those students who have the potential to succeed if they are provided with the 
appropriate supportive teaching and learning environments. 

There are variations of these extended degree programmes within and between institutions. Some 
programmes do the first year over two years while others use the first year as a foundation year. 
Essentially these extended degree models all subscribe to the idea of 'more time on task' to achieve 
their goals of providing a more appropriate teaching and learning environment. The problem 
however is that while students may be able to cope with the reduced curriculum in the first year, the 
second year then becomes the barrier. In "A proposal for undergraduate reform in South Africa: The 
case for a flexible curriculum structure" (Council on Higher Education 2013) a more bold approach is 
proposed. The approach that is proposed here is to go beyond seeing the first year as needing 
special attention and to focus instead on the entire curriculum programme. This is done for example 
by spreading the three-year curriculum over four years and the four-year curriculum programme 
over five years. In both conceptualizations of the extended foundation programme, whether it is the 
first year or the entire programme that is extended, the placement issue is of key importance. 

The South African graduation and dropout rates are of concern but even more alarming are the 
racialized patterns of these rates (Scott et. al. 2007, 16). The current admission requirements and 
curricula appear to reproduce the historical apartheid patterns. Higher education policies and 
practices need to mitigate against the racialized schooling and the manner in which racial 
classifications continue to play out in terms of higher education participation, graduation and drop¬ 
out rates. 

This paper has argued that the need for extended curriculum provision in the higher education 
landscape is necessary and should continue into the foreseeable future. It is necessary to use a 
means to determine which of the extended curriculum or regular programmes are best suited to 
individual students. While norm-referenced assessments may be statutory requirement for higher 
education study, criterion-referenced assessments such as the National Benchmark Tests may play a 
bigger role in improving graduation rates and addressing drop-out rates, by providing the kind of 
information which is needed to make decisions about which curricular route best suits students. 
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